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In June 2006, the Commission adopted an Accountability Framework for Higher Education. The 
framework consisted of four goals and 17 measures for evaluating the performance of public higher 
education in California. The framework was developed in consultation with the Commission’s Ac- 
countability Technical Advisory Committee that consisted of representatives from the public higher 
education systems and researchers who had worked on accountability issues. 

After the Commission developed the initial framework, staff produced reports for each of the perform- 
ance measures, as well as a final summary report of the overall findings from the performance assess- 
ment process. This was the first effort of its ki nd for the Commission. The initial series of reports pre- 
sented a variety of information on the affordability of postsecondary education, access to colleges and 
universities, and success and progress of students. In addition, the first year of reporting offered insight 
to how well the framework measured the performance of California’s higher education system and what 
needs to be done to make it more useful. 

Some of the key lessons learned in the initial year of reporting were as follows: 

• Statewide goals must strike a balance between clearly defining the desired direction for higher edu- 
cation performance and being too prescriptive or narrowly stated. 

• Some goals may innately be in conflict with one another. Successful performance in one area may 
contribute to less success in another area. 

• Performance indicators should be few in number. Measured outcomes should reasonably be attrib- 
uted as a direct result of higher education performance. 

• A successful framework and reporting is dependent on reliable and complete data sources. Data 
proved to be an obstacle in reporting on some measures. 

The Commission is refining its performance framework. Developing this system will evolve each year 
as goals and measures are refined and staff develop a greater understanding of how the available data 
can be shaped into useful performance measures. The Commission strives to produce a fair and precise 
instrument of evaluation that will enable it to provide useful information to policymakers and guide 
beneficial change in public higher education. 



Refinements to the Measures and Goals 

Establishing goals is the cornerstone of any effort to measure performance. A concise statement of 
goals provides a universal understanding of what the state is seeking from its higher education system. 
Goals for higher education are inherently intertwined, and defining them clearly is a challenge in per- 
formance assessment. An improvement in performance in one area may result in a decline in progress 
toward another goal. In an environment of limited resources, trade-offs in goal attainment may be un- 
avoidable. Performance assessment must balance priorities and goals should not be established or 
evaluated independently of one another. For this reason, a simple framework for performance meas- 
urement will yield the best results with minimal overlap of goals. 
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Performance measurement works best when operating under a few basic principles. Primarily, it is im- 
portant to minimize the number of measures that correspond to each goal and to use measures that pro- 
vide the strongest indication of progress toward goal attainment. Measures will vary slightly between 
the higher education systems to account for different institutional missions but are intended to collec- 
tively provide an improved understanding of how higher education is functioning from a statewide per- 
spective. 



The Revised Performance Assessment Framework 

The goals in the revised framework do not differ greatly from the original goals. The framework is 
evolving into clearer and more concise statements of desired outcomes. The performance indicators 
have been revised to include new measures that align more closely with these goals, and to drop former 
measures that proved to have little utility in the assessment process. Other measures were dropped from 
the framework due to data limitations or unreliability. 

Each of the four areas for assessment is identified as a question and contains a goal statement identifying 
the desired outcome. The areas of assessment are Preparation, Student Success, Affordability, and Pub- 
lic Benefit. 

Preparation for postsecondary education 

Measuring the degree to which students are sufficiently prepared for higher education is a foundation for 
understanding issues of college access and student success at college. Adequate preparation for post- 
secondary education will ensure that students are provided with the basic education and skills needed to 
pursue ongoing learning, whether that learning happens in a classroom or on the job. 

Measures of preparation 

There is no shortage of possible measures for student preparation and this is the only area of the frame- 
work that will require additional study to narrow the number of performance measures identified here. 
Standardized tests administered to students from junior high through high school, college preparatory 
exams, high school course completion, graduation rates, college-going rates, and remediation rates have 
all been considered as indicators of college preparedness. However, some of the measures discussed 
will, inevitably, exclude some portion of the relevant population. For example, college preparatory ex- 
ams, Advanced Placement test passage, and university remediation rates evaluate only the performance 
of college-bound students. Problems with other measures — such as math proficiency exams taken in 
the 7th grade or the California High School Exit Exam (CAHSEE) that can be taken multiple times in 
different years by the same student — present complications for meaningful data analysis. Therefore, 
Commission staff proposes the following measures for consideration with the intention of further 
evaluation to develop precise methods for interpretation. 
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Additional comments on specific measures 
English Proficiency — A previous Commission 
study found that English proficiency has a signifi- 
cant impact on incomes, and is also a predictor of 
the ability to succeed in postsecondary education. 

The California Achievement Test (CAT) and the 
California Standards Test (CST) in Eng- 
lish/Language Arts are possible measures for gaug- 
ing levels of English proficiency. Commission staff 
intends to review data for the CAT and CST more 
closely to determine which exam would provide the 
most reliable data regarding student preparation. 

CAHSEE Scores — Students can take the CAHSEE 
exam in the 10th, 11th, or 12th grade. They can take 
the exam more than once, and the results data cannot 
be cross-referenced by unique student identifying 
numbers. The best method for evaluating this meas- 
ure is to pick only one grade level to analyze. 

University Remediation — This measure will ex- 
clude many of the students who attend high school 
because it examines only preparation of college- 
going students. Nonetheless, it is an important 
measure because it provides an indication of the 
alignment between secondary and postsecondary 
education and can reveal if the top one-third of high 
school graduates are prepared for university-level 
instruction. It may also point out inconsistency in 
what is measured by high schools, through the 
CAHSEE and CST scores and what is measured by 
the college proficiency exams (the assessments that 
result in students being placed in remedial courses). 

Student success 

For the vast majority of students entering college, 
the ultimate desired outcome is to earn a degree or 
certificate, or to transfer to another institution where that outcome will be achieved. Significant finan- 
cial investment is made in each student enrollment, by the individual and their families, and also by the 
state. Those investments are maximized by successful degree completion. In addition to ensuring that 
more students are succeeding, it is critical to the state’s social and economic welfare that equity in suc- 
cess rates exists across racial/ethnic groups, socioeconomic levels, and gender. 



Preparation Question 

Are high school students receiving sufficient 
preparation to enter postsecondary education 
with the basic skills needed for success and 
continued lifetime learning? 

Preparation Goal 

Progress in elevating levels of basic math and 
English proficiency and attainment of high 
school diploma; success in narrowing the 
achievement gap among racial/ethnic groups, 
socioeconomic levels, and gender. 

Measures 

Scores on English/Language Arts CAT and CST 

CAHSEE Scores 

High school graduation rate 

Percent of students completing a-g 
requirements 

Percent of students taking the SAT 

Percent of students taking one or more 
Advanced Placement courses 
Number of Advanced Placement courses taken 
University remediation 

Disaggregated by 

High school API 
Income 
Race/Ethnicity 
Gender 
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Measures of Student Success 

The performance indicators used to measure student 
success are more evident than in the other performance 
areas. Student persistence, completion and transfer 
rates, and time-to-degree are universally accepted 
measures of performance in higher education. In mov- 
ing forward with the framework, measures for student 
success will remain the same, but the method by which 
staff evaluates the measures will evolve. 

Previous reports examined each measure of student 
success independently of the other measures. In future 
evaluation of student success, a more detailed picture 
of performance would be achieved by selecting cohorts 
and identifying persistence, completion and transfer, 
and time-to-degree rates using longitudinal analysis. 

By moving away from “snapshot” analysis, and fol- 
lowing a cohort over six years, analysts will be in a 
better position to not only identify trends, but to under- 
stand why trends might be occurring. It is important 
that all measures are disaggregated by race/ethnicity, 
gender, and income to offer a complete picture of how 
student success varies across demographic groups. 

Additional comments on specific measures 
Direct college-going rates — The last round of per- 
formance measurement used college-going rates to 
measure student preparedness for college. But because 
of the numerous factors that can affect college going, 
this is not a measure of student preparation. Rather, it represents the combined effect of student prepara- 
tion, actual and perceived cost of college attendance, and student choice. College-going rates will re- 
main a tool of performance measurement as a starting point for measuring the progress of cohort groups 
in postsecondary education. 

Persistence rates — A preliminary study of student persistence, using the Commission’s student data, is 
needed to determine the points in a college career, which may differ between systems, when students are 
most likely to drop out. Following that determination, the measuring point for persistence rates will be 
set. 

Time-to-certificate, transfer, or degree — Staff’s previous evaluation of time-to-degree is expanded to 
measure the length of time it takes to achieve one of three outcomes at the community colleges and one 
outcome at the university systems. Community college time-to-certificate, time-to-degree, or transfer 
will be evaluated on a three-year timeline. Obtaining a bachelor’s degree as an incoming transfer student 
will also be evaluated on a three-year timeline, and obtaining a bachelor’s degree as a first-time fresh- 
man will be examined on a six-year timeline. The three- and six-year timelines reflect 150% of the 
normative time-to-degree. 



Student Success Question 

Are students of all racial/ethnic and socioeco- 
nomic backgrounds achieving comparable levels 
of success in college persistence and de- 
gree/certificate completion? 

Student Success Goal 

Historically underrepresented students will 
achieve progress toward equitable levels of suc- 
cess as those student groups which have always 
maintained higher levels of success. 

Measures 

Direct college-going 

Time to community college certificate, degree, 
or transfer 

Persistence 

Degree completion / time-to-degree 

Disaggregated by 

Income levels 
Race/ethnicity 
Gender 

Full-time/part-time status 
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Affordability 

Although California institutions of public higher education remain some of the most affordable in the 
nation in terms of fees, the overall cost of attendance has ballooned considerably in recent decades. Cit- 
ing extraneous factors and declining General Fund support, the systems have justified increases in tui- 
tion and fees as a means to cover increasing costs for competitive faculty and administrative salaries, 
new classroom and laboratory technology, and updated living and recreational facilities. Still, tuition 
and fees are not the only cost factors squeezing students and their families. The sky-high cost of hous- 
ing in California accounts for a lion’s share of the overall cost of attending college. Due to a variety of 
factors beyond the cost of tuition and fees, making 
college more affordable boils down to only a few 
plausible solutions: increase the availability of fi- 
nancial aid to lower and middle-income students, 
provide students with a structured guidance path to- 
ward timely degree completion, and reduce underly- 
ing costs by efficiencies in operations so that college 
and universities can deliver their service at less cost. 

Measures of Affordability 

Measuring college affordability is challenging and 
complex, particularly when addressing questions of 
why cost of attendance is becoming such of a burden 
to families. There are a multitude of factors, such as 
housing, food, and transportation, that are not related 
to fees but create the majority of college-related ex- 
penses. 

Perhaps the greatest complexity to the affordability 
issue is in understanding the financial aid system 
and determining how financial awards impact stu- 
dents and families from varying income back- 
grounds. Additional obstacles to analyzing this is- 
sue are incomplete income data for students and in- 
complete data on all types of borrowing for college. 

Any debt analysis will likely underestimate levels of 
borrowing due to the many lending sources available 
to students and families for which reliable data are 
not readily retrievable. 

Additional comments on specific measures 
Unmet financial need and total cost of attendance — These measures will be evaluated by income cate- 
gories to determine how much students from various income levels will pay after financial aid packages, 
including loans, have been applied to overall costs. 

Average indebtedness upon completion — This measure will be examined in combination with student 
enrollment status, whether a student attends on a full-time or part-time basis. Students may feel that 
they are avoiding excess debt by working more and attending school part-time, thus extending their 
time-to-degree. Is there evidence that prolonging degree completion by attending part time actually di- 
minishes levels of student indebtedness and affects student success? 



Affordability Question 

Are the net costs of attending higher education 
institutions in California at reasonable and sta- 
ble levels so as to preserve access to all stu- 
dents who desire a postsecondary education? 

Affordability Goal 

Student fees and other costs of college remain 
stable and manageable, with changes tied to 
general cost-of-living; students graduate with 
reasonable and manageable levels of debt that 
do not impede on personal career choices or 
continued education. 

Measures 

Unmet financial need 

Total cost of attendance 

Average indebtedness upon completion 

Disaggregated by 

Family income levels 
Segment 

Enrollment status 
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Public benefit 

The missions of the higher education systems differ considerably and contribute to the public benefit in 
varying ways. The University of California fulfills a mission of “discovering and advancing knowl- 
edge” by fostering intellectual enlightenment and providing research and technology innovation to sup- 
port the state’s economy. The California State University also functions to advance knowledge and 
learning and offers instruction of applied disciplines. Many CSU graduates fill positions in areas of the 
high workforce needs, including the nursing and teaching professions. The community colleges have 
significant diversity in mission, providing open access and affordable course offerings to students seek- 
ing transfer to a baccalaureate institution, career 
training leading to a certificate, adult basic skills 
education, economic development, and arts and rec- 
reation. In light of the various benefits higher edu- 
cation provides, many of which are difficult to 
measure, the challenge in assessment of public bene- 
fit is to account for the numerous missions of public 
higher education and to use measures that are easily 
quantifiable and can reasonably be attributed to the 
efforts of postsecondary education. 

Measures of Public Benefit 

A well-educated citizenry provides many benefits to 
the community and is fundamental to a healthy de- 
mocratic society. Colleges and universities can have 
substantial impact in the areas of workforce devel- 
opment, research and technology innovation, and 
continued adult education. In addition they provide 
numerous intangible public benefits, such as acting 
as forums for growth of ideas, cultural enrichment, 
and community building. 

The complexity in measuring the public benefit of 
higher education is two-fold: First, the crafting of 
public benefit performance indicators must consider 
the diverse missions of the university systems and 
their contributions to the state; and second, the need 
to demonstrate a direct connection between the outcomes and the impact of postsecondary education as 
opposed to other social, cultural or economic activities that have public benefit outcomes. Given those 
considerations, staff recommends only two performance measures related to fulfilling the state’s work- 
force needs. While workforce outcomes are incomplete in measuring the full scope of benefit provided 
by the systems of higher education, they are easily quantifiable and relate to higher education’s primary 
statewide utility. 

Additional comments on specific measures 

Degrees awarded in areas of workforce need — Some degrees, such as Nursing, are very specific in 
their instruction; graduates in such disciplines are intended to enter a narrowly defined field. However, 
most occupations are more flexible and diverse in the degree discipline that can fill a job position. 
Commission staff has developed the Higher Education Workforce Allocation System (HEWAS), using 
data from the Bureau of Labor Statistics, which matches the many disciplines that can feed into a single 



Public Benefit Question 

Is higher education responding to the social and 
economic needs of the state and contributing to 
California’s overall advancement and well-being? 

Public Benefit Goal 

Institutions of higher education achieve a rea- 
sonable balance of preparation for the work- 
force to ensure economic well-being and foster- 
ing personal, societal, and cultural growth. 

Measures 

Degrees awarded in areas of identified 
workforce need 

Degrees awarded as a percent of college-age 
population 

Percent of workers with degrees working in 
non-degree Jobs 

Disaggregated by 

Segment 
Age group 
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occupation; and conversely, the many occupations that can be entered following study in a single disci- 
pline. HEWAS will be an important asset in evaluating the connection between degrees awarded and 
workforce need. 

Degrees awarded as a percent of college-age population — A recent report by the Western Interstate 
Commission for Higher Education (WICHE) determined that educational attainment is declining by 
generation. This measure will evaluate California’s progress toward ensuring that younger generations 
are advancing in educational attainment for the future benefit of California’s economy. 



Next Steps 

A few measures identified in this paper will require preliminary analysis in the coming months to nar- 
row which measures are best for determining progress toward higher education goals. Commission staff 
will conduct a statistical analysis to resolve the greatest academic predictors of student preparation for 
college, and incorporate such measures into the framework. 

In the coming months, Commission staff will examine measures previously studied, specifically relating 
to the goals of affordability and student success, and assess how and why certain trends are occurring. 
The June 2008 Commission agenda will include a report that will look at trends in affordability and col- 
lege costs over several decades to determine the extent to which college is becoming a greater financial 
burden to families across various income levels. 

The Commission intends to move forward with performance assessment of public higher education us- 
ing these goals, which are similar to the previous goals but stated more precisely, and narrowing the 
number of corresponding performance measures. The revised framework offers greater simplicity while 
providing the same degree of robustness in evaluation. After analysis in the coming months to deter- 
mine best methods for assessment and formal adoption of the proposed measures, staff will begin the 
next round of performance assessment. 
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